{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to TensorBoard \n",
    "\n",
    "(You can also read this article on our website, [Easy-TensorFlow](http://www.easy-tensorflow.com/basics/introduction-to-tensorboard))\n",
    "\n",
    "In the first post, we discussed the advantages of TensorFlow. They were mainly flexibility and visualization. Imagine if you can visualize whats happening in the code (in this case code represents the computational graph that we create for a model), it would be so convenient to deeply understand and observe the inner workings of the graph. Not just that, it also helps in fixing things that are not working the way they should. TensorFlow provides a way to do just that using TensorBoard!\n",
    "\n",
    "__TensorBoard__ is a visualization software that comes with any standard TensorFlow installation. In Google’s words: “The computations you'll use TensorFlow for many things (like training a massive deep neural network) and they can be complex and confusing. To make it easier to understand, debug, and optimize TensorFlow programs, we've included a suite of visualization tools called TensorBoard.”\n",
    "\n",
    "TensorFlow programs can range from a very simple to super complex problems (using thousands of computations), and they all have two basic components, Operations, and Tensors. As explained in the previous tutorials, the idea is that you create a model that consists of a set of operations, feed the data into the model and the tensors will flow between the operations until you get an output tensor i.e., your result. TensorBoard provides us with a suite of web applications that help us to inspect and understand the TensorFlow runs and graphs. Currently, it provides five types of visualizations: scalars, images, audio, histograms, and graphs.\n",
    "\n",
    "When fully configured, TensorBoard window will look something like:\n",
    "\n",
    "<img src=\"files/files/3_1.png\" width=\"500\" height=\"1000\" >\n",
    "\n",
    "___Fig. 1. ___ TensorBoard appearance\n",
    "\n",
    "\n",
    "TensorBoard was created as a way to help us understand the flow of tensors in your model so that we can debug and optimize it. It is generally used for two main purposes:\n",
    "\n",
    "__1. Visualizing the Graph__\n",
    "\n",
    "__2. Writing Summaries to Visualize Learning__\n",
    "\n",
    "We'll cover the main usages of TensorBoard in this tutorial. Learning to use TensorBoard early and often will make working with TensorFlow much more enjoyable and productive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Visualizing the Graph\n",
    "\n",
    "While powerful, TensorFlow computation graphs can become extremely complicated. Visualizing the graph can help us understand and debug it. Here's an example of the visualization at work from TensorFlow website.\n",
    "\n",
    "<img src=\"files/files/3_2.gif\" width=\"500\" height=\"1000\" >\n",
    " \n",
    "___Fig. 2. ___ Visualization of a TensorFlow graph (Source: TensorFlow website)\n",
    "\n",
    "To make our TensorFlow program __TensorBoard-activated__, we need to add some lines of code. This will export the TensorFlow operations into a file, called __event file__ (or event log file). TensorBoard is able to read this file and give some insights of the model graph and its performance.\n",
    "\n",
    "Now let's write a simple TensorFlow program and visualize its computational graph with TensorBoard.\n",
    "\n",
    "### Example 1:\n",
    "Let's create two constants and add them together. Constant tensors can be defined simply by their value:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# create graph\n",
    "a = tf.constant(2)\n",
    "b = tf.constant(3)\n",
    "c = tf.add(a, b)\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    print(sess.run(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To visualize the program with TensorBoard, we need to write log files of the program. To write event files, we first need to create a __writer__ for those logs, using this code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "writer = tf.summary.FileWriter([logdir], [graph])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where __[logdir]__ is the folder where we want to store those log files. We can also choose [logdir] to be something meaningful such as './graphs'. The second argument __[graph]__ is the graph of the program we're working on. There are two ways to get the graph:\n",
    "1. Call the graph using __tf.get_default_graph()__, which returns the default graph of the program\n",
    "2. set it as __sess.graph__ which returns the session's graph (note that this requires us to have a session created).\n",
    "\n",
    "Lets take a look at both ways in the following example; however, the second way is more common. Either way, make sure to create a writer only after defining the graph. Otherwise, the graph visualized on TensorBoard would be incomplete.\n",
    "\n",
    "Let's add the writer to the first example and visualize the graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create graph\n",
    "a = tf.constant(2)\n",
    "b = tf.constant(3)\n",
    "c = tf.add(a, b)\n",
    "\n",
    "# creating the writer out of the session\n",
    "# writer = tf.summary.FileWriter('./graphs', tf.get_default_graph())\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # or creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    print(sess.run(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if we execute this code, TensorFlow creates a directory inside your current directory (beside your Python code file) which contains the __event file__.\n",
    "\n",
    "<img src=\"files/files/3_3.png\" width=\"300\" height=\"600\" >\n",
    "\n",
    "___Fig. 3. ___ Created directory which contains the event file\n",
    "\n",
    "\n",
    "Next, to visualize the graph, we need to go to Terminal and make sure that the present working directory is the same as where we ran our Python code. For example, here we can switch to the directory using the commands:\n",
    "\n",
    "$ cd ~/Desktop/tensorboard\n",
    "\n",
    "Then run:\n",
    "\n",
    "$ tensorboard --logdir=\"./graphs\" --port 6006\n",
    "\n",
    "Replace './graphs' with the name of the directory in case you choose to name it something else. This will generate a link on the command line. Control click (ctrl+left) the link to open the TensorBoard window, TensorBoard uses the web browser to show us the visualizations (or simply copy it into your browser or just open your browser and go to http://localhost:6006/). The link will direct us to the TensorBoard page, it should look similar to:\n",
    "\n",
    "<img src=\"files/files/3_4.png\" width=\"500\" height=\"1000\" >\n",
    "\n",
    "___Fig. 4. ___ TensorBoard page visualizing the graph generated in Example 1\n",
    "\n",
    "The graph in the above picture shows us the various parts of our model. “Const” and “Const_1” in the graph correspond to a and b, and the node “Add” corresponds to c. The names given in the code (a, b, and c) are just __Python-names__, they only help us with the access while writing the code. The names mean nothing to the TensorFlow and TensorBoard. To make TensorBoard understand the names of our ops, we have to explicitly name them. \n",
    "\n",
    "Let's modify the code one more time to add the names:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create graph\n",
    "a = tf.constant(2, name=\"a\")\n",
    "b = tf.constant(3, name=\"b\")\n",
    "c = tf.add(a, b, name=\"addition\")\n",
    "\n",
    "# creating the writer out of the session\n",
    "# writer = tf.summary.FileWriter('./graphs', tf.get_default_graph())\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # or creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    print(sess.run(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"files/files/3_5.png\" width=\"500\" height=\"1000\" >\n",
    "\n",
    "___Fig. 5. ___ TensorBoard page visualizing the graph generated in Example 1 with modified names\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__*Note:__ If we run our code several times with the same [logdir], multiple event files will be generated in our [logdir]. TF will only show the latest version of the graph and display the warning of multiple event files. The warning can be removed by deleting the event files that we no longer need or else we can save them in a different [logdir] folder.\n",
    "\n",
    "\n",
    "## 2. Writing Summaries to Visualize Learning\n",
    "\n",
    "So far we only focused on how to visualize the graph in TensorBoard. Remember the other types of visualizations mentioned in the earlier part of the post that TensorBoard provides (scalars, images and histograms). In this part, we are going to use a special operation called __summary__ to visualize the model parameters (like weights and biases of a neural network), metrics (like loss or accuracy value), and images (like input images to a network). \n",
    "\n",
    "__Summary__ is a special operation TensorBoard that takes in a regular tensor and outputs the summarized data to your disk (i.e. in the event file). Basically, there are three main types of summaries:\n",
    "\n",
    "__1. tf.summary.scalar:__ used to write a single scalar-valued tensor (like a classificaion loss or accuracy value)\n",
    "\n",
    "__2. tf.summary.histogram:__ used to plot histogram of all the values of a non-scalar tensor (can be used to visualize weight or bias matrices of a neural network)\n",
    "\n",
    "__3. tf.summary.image:__ used to plot images (like input images of a network, or generated output images of an autoencoder or a GAN)\n",
    "\n",
    "In the following sections, let's go through each of the above mentioned summary types in more detail.\n",
    "\n",
    "\n",
    "### 2.1. tf.summary.scalar:\n",
    "It's for writing the values of a scalar tensor that changes over time or iterations. In the case of neural networks (say a simple network for classification task), it's usually used to monitor the changes of loss function or classification accuracy.\n",
    "\n",
    "Let's run a simple example to understand the point.\n",
    "\n",
    "### Example 2:\n",
    "Randomly pick 100 values from a standard Normal distribution, _N(0, 1)_, and plot them one after the other.\n",
    "\n",
    "One way to do so is to simply create a variable and initialize it from a normal distribution (with mean=0 and std=1), then run a for loop in the session and initialize it 100 times. The code will be as follows and the required steps to write the summary is explained in the code:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done with writing the scalar summary\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create the scalar variable\n",
    "x_scalar = tf.get_variable('x_scalar', shape=[], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "\n",
    "# ____step 1:____ create the scalar summary\n",
    "first_summary = tf.summary.scalar(name='My_first_scalar_summary', tensor=x_scalar)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # ____step 2:____ creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    for step in range(100):\n",
    "        # loop over several initializations of the variable\n",
    "        sess.run(init)\n",
    "        # ____step 3:____ evaluate the scalar summary\n",
    "        summary = sess.run(first_summary)\n",
    "        # ____step 4:____ add the summary to the writer (i.e. to the event file)\n",
    "        writer.add_summary(summary, step)\n",
    "    print('Done with writing the scalar summary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pull up TensorBoard and checkout the result. Like before, you need to open terminal and type:\n",
    "\n",
    "$ tensorboard --logdir=\"./graphs\" --port 6006\n",
    "\n",
    "Here _\"./graphs\"_ is the name of the directory we saved the event file to. In TensorBoard, we find a new tab named __\"scalars\"__ next to the __\"graphs\"__ tab earlier discussed (compare Fig. 5 with Fig. 6). The whole window looks like:\n",
    "\n",
    "<img src=\"files/files/3_6.png\" width=\"500\" height=\"1000\" >\n",
    "\n",
    "___Fig. 6. ___ TensorBoard page visualizing the written scalar summary.\n",
    "\n",
    "In the figure, the plot panel is under the name \"My_first_scalar_summary\", the same name that we defined in our code. The x-axis and y-axis shows the 100 steps and the corresponding values (random values from a standard normal dist.) of the variable respectively. \n",
    "\n",
    "\n",
    "### 2.2. tf.summary.histogram:\n",
    "\n",
    "Histogram comes in handy if we want to observe the change of a value over time or iterations. It's used for plotting the histogram of the values of a non-scalar tensor. This provides us a view of how the histogram (and the distribution) of the tensor values change over time or iterations. In the case of neural networks, it's commonly used to monitor the changes of weight and biase distributions. It's very useful in detecting irregular behavior of the network parameters (for example, when our weights explode or shrink abnormally). \n",
    "\n",
    "Now let's go back to our previous example and add the histogram summary to it.\n",
    "\n",
    "### Example 3:\n",
    "Continue the previous example by adding a matrix of size 30x40, whose entries come from a standard normal distribution. Initialize this matrix 100 times and plot the distribution of its entries over time.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done writing the summaries\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create the variables\n",
    "x_scalar = tf.get_variable('x_scalar', shape=[], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "x_matrix = tf.get_variable('x_matrix', shape=[30, 40], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "\n",
    "# ____step 1:____ create the summaries\n",
    "# A scalar summary for the scalar tensor\n",
    "scalar_summary = tf.summary.scalar('My_scalar_summary', x_scalar)\n",
    "# A histogram summary for the non-scalar (i.e. 2D or matrix) tensor\n",
    "histogram_summary = tf.summary.histogram('My_histogram_summary', x_matrix)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # ____step 2:____ creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    for step in range(100):\n",
    "        # loop over several initializations of the variable\n",
    "        sess.run(init)\n",
    "        # ____step 3:____ evaluate the merged summaries\n",
    "        summary1, summary2 = sess.run([scalar_summary, histogram_summary])\n",
    "        # s____step 4:____ add the summary to the writer (i.e. to the event file) to write on the disc\n",
    "        writer.add_summary(summary1, step)\n",
    "        # repeat steps 4 for the histogram summary\n",
    "        writer.add_summary(summary2, step)\n",
    "    print('Done writing the summaries')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In TensorBoard, two new tabs are added to the top menu: \"Distributions\" and \"Histograms\". The results will be as follows:\n",
    "\n",
    "<img src=\"files/files/3_7.png\">\n",
    "\n",
    "___Fig. 7. ___ (a) scalar summary, (b) distribution and (c) histogram of the values of the 2D-tensor over 100 steps\n",
    "\n",
    "\n",
    "In the figure, the \"Distributions\" tab contains a plot that shows the distribution of the values of the tensor (y-axis) through steps (x-axis). You might ask what are the light and dark colors? \n",
    "\n",
    "The answer is that each line on the chart represents a percentile in the distribution over the data. For example, the bottom line (the very light one) shows how the minimum value has changed over time, and the line in the middle shows how the median has changed. Reading from top to bottom, the lines have the following meaning: [maximum, 93%, 84%, 69%, 50%, 31%, 16%, 7%, minimum]\n",
    "\n",
    "These percentiles can also be viewed as standard deviation boundaries on a normal distribution: [maximum, μ+1.5σ, μ+σ, μ+0.5σ, μ, μ-0.5σ, μ-σ, μ-1.5σ, minimum] so that the colored regions, read from inside to outside, have widths [σ, 2σ, 3σ] respectively.\n",
    "\n",
    "Similarly, in the histogram panel, each chart shows temporal \"slices\" of data, where each slice is a histogram of the tensor at a given step. It's organized with the oldest timestep in the back, and the most recent timestep in front.\n",
    "\n",
    "You can easily monitor the values on the histograms at any step. Just move your cursor on the plot and see the x-y values on the histograms (Fig8 (a)). You can also change the Histogram Mode from \"offset\" to \"overlay\" (see Fig. 8- (b)) to see the histograms overlaid with one another.\n",
    "\n",
    "\n",
    "<img src=\"files/files/3_8.png\">\n",
    "\n",
    "___Fig. 8. ___ (a) monitor values on the histograms, (b) overlayed histograms \n",
    "\n",
    "\n",
    "As mentioned in the code, we need to run every summary (e.g. __sess.run([scalar_summary, histogram_summary])__) and then use our writer to write each of them to the disk. In practice, we can use any number of summaries to track different parameters in our model. This makes running and writing the summaries extremly inefficient. The way around it is to merge all summaries in our graph and run them at once inside your session. This can be done with __tf.summary.merge_all()__ method. Let's add it to Example 3, the code changes as follows:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done writing the summaries\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create the variables\n",
    "x_scalar = tf.get_variable('x_scalar', shape=[], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "x_matrix = tf.get_variable('x_matrix', shape=[30, 40], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "\n",
    "# ____step 1:____ create the summaries\n",
    "# A scalar summary for the scalar tensor\n",
    "scalar_summary = tf.summary.scalar('My_scalar_summary', x_scalar)\n",
    "# A histogram summary for the non-scalar (i.e. 2D or matrix) tensor\n",
    "histogram_summary = tf.summary.histogram('My_histogram_summary', x_matrix)\n",
    "\n",
    "# ____step 2:____ merge all summaries\n",
    "merged = tf.summary.merge_all()\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # ____step 3:____ creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    for step in range(100):\n",
    "        # loop over several initializations of the variable\n",
    "        sess.run(init)\n",
    "        # ____step 4:____ evaluate the merged summaries\n",
    "        summary = sess.run(merged)\n",
    "        # ____step 5:____ add summary to the writer (i.e. to the event file) to write on the disc\n",
    "        writer.add_summary(summary, step)\n",
    "    print('Done writing the summaries')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2. tf.summary.image:\n",
    "As the name implies, this type of summary is used for writing and visualizing tensors as images. In the case of neural networks, this is usually used for tracking the images that are either fed to the network (say in each batch) or the images generated in the output (such as the reconstructed images in an autoencoder; or the fake images made by the generator model of a Generative Adverserial Network). However, in general, this can be used for plotting any tensor. For example, we can visualize a weight matrix of size 30x40 as an image of 30x40 pixels.\n",
    "\n",
    "An image summary can be created using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.summary.image(name, tensor, max_outputs=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Where __name__ is the name for the generated node (i.e. operation), tensor is the desired tensor to be written as an image summary (we will talk about its shape shortly), and max_outputs is the maximum number of elements from __tensor__ to generate images for. but... what does it mean? The answer lies in the the shape of the __tensor__.\n",
    "\n",
    "The __tensor__ that we feed to tf.summary.image must be a 4-D tensor of shape __[batch_size, height, width, channels]__ where batch_size is the number of images in the batch, the height and width determine the size of the image and finally, the channels are:\n",
    "1: for Grayscale images.\n",
    "3: for RGB (i.e. color) images.\n",
    "4: for RGBA images (where A stands for alpha; see [RGBA](https://en.wikipedia.org/wiki/RGBA_color_space)).\n",
    "\n",
    "Let's look at a very simple example to get the underlying idea.\n",
    "\n",
    "### Example 4:\n",
    "Let's define two variables: \n",
    "1. Of size 30x10 as 3 __grayscale__ images of size 10x10\n",
    "2. Of size 50x30 as 5 __color__ images of size 10x10\n",
    "\n",
    "and plot them as images in TensorBoard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done writing the summaries\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.reset_default_graph()   # To clear the defined variables and operations of the previous cell\n",
    "\n",
    "# create the variables\n",
    "w_gs = tf.get_variable('W_Grayscale', shape=[30, 10], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "w_c = tf.get_variable('W_Color', shape=[50, 30], initializer=tf.truncated_normal_initializer(mean=0, stddev=1))\n",
    "\n",
    "# ___step 0:___ reshape it to 4D-tensors\n",
    "w_gs_reshaped = tf.reshape(w_gs, (3, 10, 10, 1))\n",
    "w_c_reshaped = tf.reshape(w_c, (5, 10, 10, 3))\n",
    "\n",
    "# ____step 1:____ create the summaries\n",
    "gs_summary = tf.summary.image('Grayscale', w_gs_reshaped)\n",
    "c_summary = tf.summary.image('Color', w_c_reshaped, max_outputs=5)\n",
    "\n",
    "# ____step 2:____ merge all summaries\n",
    "merged = tf.summary.merge_all()\n",
    "\n",
    "# create the op for initializing all variables\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "# launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # ____step 3:____ creating the writer inside the session\n",
    "    writer = tf.summary.FileWriter('./graphs', sess.graph)\n",
    "    # initialize all variables\n",
    "    sess.run(init)\n",
    "    # ____step 4:____ evaluate the merged op to get the summaries\n",
    "    summary = sess.run(merged)\n",
    "    # ____step 5:____ add summary to the writer (i.e. to the event file) to write on the disc\n",
    "    writer.add_summary(summary)\n",
    "    print('Done writing the summaries')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now open TensorBoard like before and switch to __IMAGES__ tab. The images should be something similar to:\n",
    "<img src=\"files/files/3_9.png\">\n",
    "\n",
    "___Fig. 9. ___ generated images in TensorBoard\n",
    "\n",
    "\n",
    "We can similarly add any other image of any size to our summaries and plot them in TensorBoard. We'll see more of such summaries in our next tutorials. Start with our [Linear Classifier](http://localhost:8888/notebooks/2_Linear_Classifier/Tutorials/2_Linear_Classifier%20_TensorBoard.ipynb) post for warm up.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I hope this post has helped you to understand the basic key features of __TensorBoard__. Thank you so much for reading! If you have any questions, feel free to leave a comment in our [webpage](http://www.easy-tensorflow.com/basics/introduction-to-tensorboard). You can also send us feedback through the [__contacts__](http://www.easy-tensorflow.com/contacts) page."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
